Generalized Resolvents and Harris Recurrence of Markov Processes
نویسندگان
چکیده
In this paper we consider a φ-irreducible continuous parameter Markov process Φ whose state space is a general topological space. The recurrence and Harris recurrence structure of Φ is developed in terms of generalized forms of resolvent chains, where we allow statemodulated resolvents and embedded chains with arbitrary sampling distributions. We show that the recurrence behavior of such generalized resolvents classifies the behavior of the continuous time process; from this we prove that hitting times on the small sets of a generalized resolvent chain provide criteria for, successively, (i) Harris recurrence ofΦ (ii) the existence of an invariant probability measure π (or positive Harris recurrence of Φ) and (iii) the finiteness of π(f) for arbitrary f .
منابع مشابه
Utilizing Generalized Learning Automata for Finding Optimal Policies in MMDPs
Multi agent Markov decision processes (MMDPs), as the generalization of Markov decision processes to the multi agent case, have long been used for modeling multi agent system and are used as a suitable framework for Multi agent Reinforcement Learning. In this paper, a generalized learning automata based algorithm for finding optimal policies in MMDP is proposed. In the proposed algorithm, MMDP ...
متن کاملWide-sense regeneration for Harris recurrent Markov processes: an open problem
Harris recurrence is a widely used tool in the analysis of queueing systems. For discrete-time Harris chains, such systems automatically exhibit wide-sense regenerative structure, so that renewal theory can be applied to questions related to convergence of the transition probabilities to the equilibrium distribution. By contrast, in continuous time, the question of whether all Harris recurrent ...
متن کاملADK Entropy and ADK Entropy Rate in Irreducible- Aperiodic Markov Chain and Gaussian Processes
In this paper, the two parameter ADK entropy, as a generalized of Re'nyi entropy, is considered and some properties of it, are investigated. We will see that the ADK entropy for continuous random variables is invariant under a location and is not invariant under a scale transformation of the random variable. Furthermore, the joint ADK entropy, conditional ADK entropy, and chain rule of this ent...
متن کاملRandom Recurrence Equations and Ruin in a Markov-dependent Stochastic
We develop sharp large deviation asymptotics for the probability of ruin in a Markov-dependent stochastic economic environment and study the extremes for some related Markovian processes which arise in financial and insurance mathematics, related to perpetuities and the ARCH(1) and GARCH(1,1) time series models. Our results build upon work of Goldie [Ann. Appl. Probab. 1 (1991) 126–166], who ha...
متن کاملRandom Recurrence Equations
We develop sharp large deviation asymptotics for the probability of ruin in a Markov-dependent stochastic economic environment and study the extremes for some related Markovian processes which arise in financial and insurance mathematics, related to perpetuities and the ARCH(1) and GARCH(1,1) time series models. Our results build upon work of Goldie [Ann. Appl. Probab. 1 (1991) 126–166], who ha...
متن کامل